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Abstract 

Our  concern  lies  in  solving  the  following  convex  optimization  prob- 
lem: 

Gp  :     minimize^     c^x 

s.t.  Ax  =  b 

X  eP, 

where  P  is  a  closed  convex  subset  of  the  n-dimensional  vector  space 
X.  We  bound  the  complexity  of  computing  an  almost-optimal  solution 
of  Gp  in  terms  of  natural  geometry-based  measures  of  the  feasible  re- 
gion and  the  level-set  of  almost-optimal  solutions,  relative  to  a  given 
reference  point  x^  that  might  be  close  to  the  feasible  region  and/or  the 
almost-optimal  level  set.  This  contrasts  with  other  complexity  bounds 
for  convex  optimization  that  rely  on  data-based  condition  numbers  or 
algebraic  measures,  and  that  do  not  take  into  account  any  a  priori  ref- 
erence point  information. 
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1      Introduction,  Motivation,  and  Main  Result 

Consider  the  following  convex  optimization  problem: 

Gp  :     z*  :=  minimum^    c^x 

s.t.  Ax  —  b 

X  e  P. 

where  P  is  a  closed  convex  set  in  the  (finite)  7i-dimensional  linear  vector  space 
X,  and  b  lies  in  the  (finite)  m-dimensional  vector  space  Y.  We  call  this  prob- 
lem linear  optimization  with  a  ground-set,  and  we  call  P  the  ground-set.  In 
practical  applications,  P  could  be  the  solution  of  box  constraints  of  the  form 
/  <  X  <  u,  a  convex  cone  C  or  perhaps  the  solution  to  network  flow  con- 
straints of  the  form  Ni  =  b.x  >  0.  However,  for  ease  of  presentation,  we  will 
make  the  following  assumption: 

Assumption  A:  P  has  an  interior,  and  {x  |  Ax  =  6}  fl  intP  /  0. 

For  e  >  0.  we  call  x  an  e-optimal  solution  of  Gp  if  x  is  a  feasible 
solution  of  Gp  that  satisfies  c^x  <  z*  +  e.  The  chief  concern  in  this  paper 
is  an  algorithm  and  associated  complexity  bound  for  computing  an  e-optimal 
solution  of  Gp. 

Let  II  •  II  be  any  norm  on  A',  and  let  B{x.  r)  denote  the  ball  of  radius  r 
centered  at  x: 

B{x,r)  :=  {u-  G  A'  |  ||u,'  —  x||  <  r}  . 

The  norm  ||  •  ||  might  be  a  problem-appropriate  norm  for  the  actual  problem 
context  at  hand.  However,  in  Section  5.1.  we  will  examine  in  detail  two  norms 
on  A'  that  arise  "naturally"'  in  association  with  the  ground-set  P. 

The  computational  engine  that  we  will  use  to  solve  Gp  is  the  barrier 
method  based  on  the  theory  of  self-concordant  barriers,  and  we  presume  that 
the  reader  has  a  general  familiarity  with  this  topic  as  developed  in  [5]  and/or 
[7],  for  example.  We  therefore  assume  that  we  have  a  7?p-self-concordant  bar- 
rier Fp[-)  for  P.  We  also  assume  that  we  have  a  i^y  ||-self-concordant  barrier 
F||  ||(-)  for  the  unit  ball: 

5(0,1)  =  {x  I  |lx||  <  1}   . 

The  work  here  is  motivated  by  a  desire  to  generalize  and  improve  sev- 
eral aspects  of  the  general  complexity  theory  for  conic  convex  optimization 
developed  by  Renegar  in  [7],  key  elements  of  which  we  now  attempt  to  sum- 
marize in  a  brief  and  somewhat  simplified  manner.  In  [7] ,  the  general  convex 
optimization  problem  Gp  is  assumed  to  be  conic,  that  is,  P  is  assumed  to 
be  closed  convex  cone  C .  and  the  data  for  the  problem  is  given  by  the  array 
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d  =  (A,b,c).  One  complexity  result  that  can  be  gleaned  from  [7]  is  as  fol- 
lows: assuming  that  Gp  has  a  feasible  solution,  there  is  an  algorithm  based 
on  interior-point  methods  that  will  compute  an  e-optimal  solution  of  G p  in 


O 


^  \  €        dist(x,  aC)       mm{s,  ||c(||}  yy 


iterations  of  Newton's  method  (see  Theorem  3.1  and  Corollary  7.3  of  [7]), 
where  we  use  the  notation  dc  to  denote  the  complexity  value  of  the  barrier  for 
the  cone  C.  Here  x  G  C  is  a  given  interior  point  of  the  cone  C  that  is  specified 
as  part  of  the  input  to  the  algorithm,  dist(x,  dC)  is  the  distance  from  x  to  the 
boundary  of  C,  \\d\\  is  the  (suitably  defined)  norm  of  the  data  ci,  and  s  is  a 
positive  scalar  that  must  be  specified  as  input  to  the  algorithm.  The  quantity 
C{d)  is  the  condition  number  of  the  data  d,  defined  as: 

Cid)  ■= ^^ (2) 

"-^^^^      mm{pp(d),po(d)}  '  ^'> 

where  pp{d), poid)  are  the  primal  and  dual  distances  to  ill-posedness,  see  [7] 
for  details  and  motivating  discussion.  {C{d)  naturally  extends  the  concept 
of  condition  number  of  a  system  of  equations  to  the  far  broader  problem  of 
conic  convex  optimization.)  The  complexity  result  (1)  is  remarkable  for  its 
breadth  and  generality,  as  well  as  for  its  reliance  on  natural  data-dependent 
concepts  imbedded  in  condition-number  theory.  In  order  to  keep  the  presen- 
tation brief,  we  have  shown  a  simplified  and  slightly  weaker  complexity  result 
in  (1)  than  the  verbatim  complexity  bound  in  [7].  Furthermore,  [7]  has  many 
other  complexity  results  related  to  conic  convex  optimization  in  finite  as  well 
as  infinite-dimensional  settings. 

While  the  significance  of  (1)  and  the  many  related  results  in  [7]  cannot 
be  overstated,  there  are  certain  issues  with  this  type  of  complexity  bound  that 
are  not  very  satisfactory.  One  issue  has  to  do  with  undue  data  dependence. 
Given  a  data  instance  d  =  {A,  6,  c)  for  Gp  and  a  nonsingular  matrix  B  and 
a  vector  n  of  multipliers,  we  can  create  an  equivalent  representation  of  the 
problem  Gp  using  the  different  data  d  —  {A,b,c)  :—  {B~^A,B~^b,c  —  A'^n). 
The  two  data  instances  d  and  d  will  generally  give  rise  to  different  complexity 
bounds  using  (1)  since  in  general  C{d)  ^  C{d),  etc.,  yet  both  data  instances 
represent  the  same  underlying  optimization  problem. 

Another  issue  with  the  condition  number  approach  is  that  the  prob- 
lem must  be  in  conic  form.  While  any  convex  optimization  problem  can  be 
transformed  to  conic  form,  such  a  transformation  might  not  be  natural  (such 
as  converting  a  quadratic  objective  to  a  linear  objective  using  a  second-order 
cone  constraint,  etc.)  or  unique  (and  so  might  further  introduce  arbitrarily 
different  data  for  the  same  original  problem).  Yet  a  third  issue  with  the  condi- 
tion number  approach  has  to  do  with  the  fact  that  the  theory  assumes  that  the 
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data  is  arising  only  in  the  linear  equation  system  and  the  objective  function, 
and  that  the  cone  C  is  fixed  independent  of  any  data.  In  this  format,  data 
used  to  defined  the  cone  is  not  accounted  for  in  the  theory. 

A  fourth  issue  has  to  do  with  the  role  of  the  starting  point  x.  The 
bound  (1)  is  not  dependent  on  or  sensitive  to  the  extent  to  which  x  might  be 
nearly  feasible  and/or  nearly  optimal.  It  would  be  nice  to  have  a  complexity 
bound  that  accounted  for  the  proximity  of  x  to  the  feasible  and/or  optimal 
solution  set. 

The  algorithm  and  analysis  presented  in  this  paper  represent  an  at- 
tempt to  overcome  the  above-mentioned  issues.  In  Sections  3  and  4,  we  develop 
and  analyze  interior-point  algorithms  FEAS  and  OPT  for  finding  a  feasible  so- 
lution x  of  Gp  and  an  e-optimal  solution  x  of  Gp,  respectively.  This  pair  of 
algorithms  and  their  complexity  analysis  depend  on  certain  geometry-based 
measures  for  analyzing  convex  optimization  problems  and  the  concept  of  a 
reference  point  x^ ,  which  we  now  discuss. 


1.1     Reference  Point  and  Interior  Point 

The  phase-I  algorithm  FEAS  requires  that  the  user  specify  two  points  as  part 
of  the  input  of  the  algorithm,  the  reference  point  x^,  and  an  interior  point 
x°  G  intF.  The  reference  point  x""  might  be  chosen  to  be  an  initial  guess  of 
a  feasible  and/or  optimal  solution,  the  solution  to  a  previous  version  of  the 
problem  (such  as  in  warm-start  methodologies),  or  the  origin  0  of  the  space 
X,  etc.  If  P  is  the  box  defined  by  the  constraints  I  <  x  <  u,  then  x''  might 
be  chosen  as  a  given  corner  of  the  box  such  as  x'"  =  /  ;  if  P  is  convex  cone 
C,  x^  might  be  chosen  to  be  the  origin  x""  =  0,  or  a  known  point  on  the 
boundary  or  the  interior  of  C,  etc.  Certain  properties  of  x'"  will  enter  into  the 
complexity  bounds  derived  herein,  particularly  related  to  the  distance  from 
x""  to  the  feasible  region,  to  the  set  of  e-optimal  solutions,  and  to  the  set  of 
nicely-interior  feasible  solutions.  There  is  no  assumption  concerning  whether 
or  not  x""  is  in  the  ground-set  P  or  satisfies  the  linear  equations  Ax  =  b. 

Algorithm  FEAS  also  requires  an  initial  point  x°  €  intP.  This  interior 
point  will  be  used  in  many  ways  to  measure  how  interior  other  points  in  P 
are.  (By  analogy,  in  linear  optimization  e  :=  (1, . . . ,  1)-^  is  used  to  measure 
the  positivity  of  other  vectors  v  by  computing  the  largest  a  for  which  v  >  ae.) 
It  will  be  desirable  for  x°  to  be  nicely  interior  to  P.  We  define: 

r  :=r(x°)  :=min{dist(x°,aP),l}   .  (3) 

Although  the  quantity  dist(x°,c>P)  will  enter  into  our  complexity  bounds  for 
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solving  Gp  through  the  quantity  t{x°),  we  do  not  require  that  we  Icnow  the 
value  of  dist(x°,  dP).  However,  for  the  two  important  classes  of  norms  that  we 
will  consider  in  Section  5.1,  we  will  show  that  d'ist{x°,dP)  >  1  which  implies 
that  t{x^)  —  1;  this  will  be  very  desirable  from  a  complexity  point  of  view. 


1.2     Phase  I  Geometry  Measure  g 

The  complexity  of  the  phase-I  algorithm  FEAS  will  be  bounded  by  the  follow- 
ing geometric  measure  which  we  denote  by  g  : 


max{||x  —  x^W,  1} 

(4) 


min{r,  1} 
s.t.  Ax  =  b 

B{x,r)  C  P  . 


If  we  ignore  for  the  moment  the  "T's  in  the  numerator  and  denominator  of 
the  ratio  defining  g,  (4)  could  be  re-written  as: 


X  —  X 


g  :—  mimmunij. 


dist(x,(9P) 
s.t.  Ax  =  b  ^^^ 

xeP  . 

and  so  g  (or  g)  measures  the  extent  to  which  x'"  is  close  to  an  interior  feasible 
solution  X  that  is  itself  not  too  close  to  the  boundary  of  P,  and  g  (or  g)  is 
smaller  to  the  extent  that  x'"  is  close  to  feasible  solutions  x  G  intP  that  are 
themselves  far  from  the  boundary  of  P. 

The  ratios  defining  g  and  g  arise  naturally  in  the  complexity  of  the 
elUpsoid  algorithm  applied  to  the  problem  of  finding  a  feasible  solution  of 
Gp.  If  one  were  to  initiate  the  ellipsoid  algorithm  at  the  ball  centered  at  the 
reference  point  x^  with  a  radius  given  by  ||x''  —  x\\  +  f  (where  {x,f)  are  an 
optimal  solution  of  (5)),  then  it  is  easy  to  see  that  a  suitably  designed  version 
of  the  ellipsoid  method  would  compute  a  feasible  solution  of  Gp  in  O  (n^  ln(5)) 
iterations,  under  the  presumption  that  the  norm  ||  •  ||  is  ellipsoidal.  (We  refer 
readers  to  [4]  for  an  excellent  treatment  of  the  ellipsoid  algorithm.)  In  the  more 
typical  context  in  continuous  optimization  where  we  do  not  have  an  a  priori 
bound  on  the  distance  from  the  feasible  region  to  the  reference  point,  there 
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is  a  natural  projective  transformation  of  tlie  problem  for  which  the  ellipsoid 
algorithm  will  compute  a  feasible  solution  of  Gp  in  O  (71^  ln(5))  iterations,  see 
Lemma  4.1  of  [2].  Therefore  5  is  a  very  relevant  geometric  measure  for  the 
Phase-I  problem  in  the  context  of  the  ellipsoid  algorithm.  Herein,  we  will  see 
that  g  is  also  relevant  for  the  complexity  of  the  Phase-I  problem  for  a  suitably 
constructed  interior-point  algorithm. 

Incidentally,  the  constants  "1"  appearing  in  the  numerator  and  denomi- 
nator of  the  ratio  defining  g  in  (4)  appear  for  the  convenience  of  the  complexity 
analysis,  and  could  be  replaced  by  any  other  positive  absolute  constants  71, 72. 


1.3     Phase  II  Geometry  Measure  D^ 

Our  complexity  analysis  of  the  phase-II  algorithm  OPT  will  rely  on  the  max- 
imum distance  from  the  reference  point  x'"  to  the  set  of  e-optimal  solutions: 

D^  :=  max  |||x  -  x""!!  |  Ax  =  b,x  e  P,  c^x  <  z'  +  el   .  (6) 

At  first  glance  it  may  seem  odd  to  maximize  rather  than  minimize  in  defin- 
ing D^.  However,  consider  the  ill-posed  case  when  z'  is  finite  but  the  set  of 
optimal  solutions  is  unbounded,  which  can  arise,  for  example,  in  semidefinite 
optimization.  Then  the  dual  feasible  region  has  no  interior,  and  so  we  would 
not  expect  to  have  an  efficient  complexity  bound  for  solving  Gp.  In  this  con- 
text, the  more  relevant  complexity  measure  is  the  maximum  distance  to  the 
e-optimal  solution  set  (which  would  be  infinite  in  this  case)  rather  than  the 
minimum  distance  (which  would  be  finite  in  this  case).  Also,  in  [1]  in  the  case 
of  conic  optimization  with  .r''  =  0,  it  is  shown  that  D^  defined  using  (6)  is 
inversely  proportional  to  the  size  of  the  largest  ball  contained  in  the  level  sets 
of  the  dual  problem,  and  so  D^  is  very  relevant  in  studying  the  behavior  of 
primal-dual  and/or  dual  interior-point  algorithms  for  conic  problems. 


1.4     Main  Result 

Theorems  3.1  and  4.1  contain  complexity  bounds  on  the  phase-I  and  phase- 
II  algorithms  FEAS  and  OPT,  respectively.  Taken  as  a  pair,  the  combined 
complexity  bound  for  the  algorithms  to  compute  an  e-optimal  solution  of  Gp 
using  the  reference  point  x''  and  the  interior-point  x''  is: 
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O 


\/^P  +  '^\\  II  In 


v 


/''^/-  +  ^llll  +  m,n{dist/x°.aP).l}   +  ll^°--^'ll    \\ 


^  +P  + A  +  max{f,l} 


(7) 


iterations  of  Newton's  method,  where  the 

s  :=  m&xlc^w  I  ||u;||  <  I,  Aw  =  0>  <  ||c||,  . 

U)  I  J  M       II 

Note  that  (7)  depends  logarithmically  on  the  phase-I  and  phase-II  geometry 
measures  g  and  D^,  the  inverse  of  the  distance  from  x°  to  the  boundary  of  P, 
as  well  as  the  distance  from  x^  to  the  reference  point  x". 

In  Section  5.1,  we  present  two  choices  of  norms  ||  ■  ||  on  A'  that  arise 
naturally  and  for  which  the  complexity  bound  (7)  simplifies  to: 


0  [/J^ln  (g  +  D,  +  dp  +  max  |-,  l|  +  ||x°  -  x' 

iterations  of  Newton's  method.  In  Section  5.2,  we  show  how  the  condition- 
number  based  complexity  bound  (1)  can  be  derived  as  a  special  case  of  Theo- 
rems 3.1  and  4.1. 


2     Summary  of  Interior-Point  Methodology 

We  employ  the  basic  theoretical  machinery  of  interior-point  methods  in  our 
analysis  using  the  theory  of  self-concordant  barrier  functions  as  articulated 
in  Renegar  [7]  and  [8],  based  on  the  theory  of  self-concordant  functions  of 
Nesterov  and  Xemirovskii  [5].  The  barrier  method  is  essentially  designed  to 
approximately  solve  a  problem  of  the  form 

OP  :      z  =  min{c^u;  |  it)  G  5}, 

where  5  is  a  compact  convex  subset  of  the  7i-dimensional  space  A',  and  c  €  A^*. 
The  method  requires  the  existence  of  a  self-concordant  barrier  function  F{w) 
for  the  relative  interior  of  the  set  5,  see  [7]  and  [5]  for  details,  and  proceeds 
by  approximately  solving  a  sequence  of  problems  of  the  form 

OP^  :      minjc-^ii;  -f-  ^iF{w)  \  w  6  relint5}, 


for  a  decreasing  sequence  of  values  of  the  barrier  parameter  fj,.   We  base  our 
complexity  analysis  on  the  general  convergence  results  for  the  barrier  method 
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presented  in  Renegar  [7],  which  are  similar  to  (but  are  more  accessible  for 
our  purposes  than)  related  results  found  in  [5].  The  barrier  method  starts 
at  a  given  point  w^  €  relint5.  The  method  performs  two  stages.  In  stage  I, 
the  method  starts  from  w°  and  computes  iterates  based  on  Newton's  method, 
ending  when  it  has  computed  a  point  w  that  is  an  approximate  solution  of 
OP^  for  some  barrier  parameter  fi  that  is  generated  internally  in  stage  I.  In 
stage  II,  the  barrier  method  computes  a  sequence  of  approximate  solutions  w'' 
of  OP^i^,  again  using  Newton's  method,  for  a  decreasing  sequence  of  barrier 
parameters  fi^  converging  to  zero.  The  goal  of  the  barrier  method  is  to  find 
an  e-optimal  solution  of  OP,  which  is  a  feasible  solution  w  of  OP  for  which 
c^w  <  z  +  e.  One  description  of  the  complexity  of  the  barrier  method  is  as 
follows: 


•  Assume  that  S  is  a  bounded  set,  and  that  w^  G  relintS  is  given.    Tlie 
barrier  method  requires 

o(>/d\n(i)  + 1^  +  ^))  (8) 

V  V        sym(u;^5)       eJJ 

iterations  of  Newton's  method  to  compute  an  e-optimal  solution  of  OP. 

In  the  above  expression,  R  is  the  range  of  the  objective  function  c^w 
over  the  set  S,  that  is,  R  =  z"'  —  2'  where 

2'  =  m\n{(Fw  I  u;  G  5}     and     2"  =  maxjc'^u'  |  w  G  5}, 

and  sym(u;,  S)  is  a  measure  of  the  symmetry  of  the  point  w  with  respect  to 
the  set  S,  and  is  defined  as 

sym(u',  S)  :=  max{^  \  y  ^  S  ^  w  -  t{y  —  w)  £  S]. 

This  term  in  the  complexity  of  the  barrier  method  arises  since  the  closer  the 
starting  point  is  to  the  boundary,  the  larger  is  the  value  of  the  barrier  function 
at  this  point,  and  so  more  effort  is  generally  required  to  proceed  from  such  a 
point. 

The  barrier  method  can  also  be  used  in  equation-solving  mode,  to  solve 
the  system: 


weS 
c^w  =  6 


(9) 


for  some  given  value  of  6.  A  description  of  the  complexity  of  the  barrier  method 
for  equation-solving  mode  is  as  follows: 
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•  Assume  tfiat  S  is  a  bounded  set  and  that  w°  G  relintS.   If  6  G  f.:',2"j, 
the  barrier  method  requires 

0{^\n\i)  + ,    „   ^^  +^—r ^^ tt]]  (10) 

iterations  to  compute  a  point  w  that  satisfies  w  E  S,  c^w  =  S.   Further- 
more, w  will  also  satisfy: 

.ymKr)>3.^|^_^^  (11) 

where  T  is  the  level  set: 

T  :=  Iw  \  w  E  S,c  w  —  5\   . 

Because  it  will  play  a  prominent  role  in  our  analysis,  we  present  a  derivation 
(11)  based  on  [5]  and  [7],  under  the  assumption  that  5  has  an  interior  and 
contains  no  line.  Let  w'^  denote  the  analytic  center  of  T,  namely 

w'^  :—  argmiUj^  {F{w)  \  w  E  T}  . 

For  w  G  S,  let  ||  ■  ||^,  denote  the  norm  induced  by  the  Hessian  H{w)  of  the 
barrier  function  F(-)  at  w,  namely  \\v\\^  :—  Jv^H{w)v,  and  let  i]{w)  denote 
the  Newton  direction  for  F{-)  at  w,  namely  ri{w)  :=  —H{w)~^VF{w).  Recall 
from  Proposition  2.3.2  of  [5]  that  all  u;  €  T  satisfy  \\w  —  w'^W^c  <  3i3  +  1. 
There  is  a  fixed  constant  7  <  1,  the  value  of  7  being  dependent  on  the  specific 
implementation  of  the  barrier  method,  such  that  the  final  iterate  u'  G  T  of 
the  barrier  method  will  satisfy  ||7?(u')  —  Ac||^  <  7  for  some  multiplier  A.  By 
taking  a  fixed  extra  number  of  Newton  steps  if  necessary,  we  can  assume  that 
7  :=  ^.  Then  from  Theorem  2.2.5  of  [8]  we  have  \\w  -  w'^\\u,  <  7  +  n^hr  <  f , 
and  so  for  all  w  G  T  we  have 


\w  —  wWw    <  \\w  —  w'^Wyij  +  Ww"^  —  w\\ 

<  (itt)  \\u'  -  w'\\^,c  +  i 

<  1(3^1)  +  ! 

<  3.5t?  +  1.25. 


(12) 


(The  second  inequality  above  follows  from  Theorem  2.1.1  of  [5].)  Furthermore, 
if  u;  satisfies  c^w  =  5  and  \\w  -  u'||u,  <  1,  then  w  €  T  (also  from  Theorem  2.1.1 
of  [5]),  and  together  with  (12)  this  then  implies  that  sym(u',r)  >  3-5^5^:1-25 ■ 
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Remark  2.1  Note  tliat  (11)  implies  that  for  any  objective  function  vector 
seX': 

maxs^w  -  s^w  <  (3.5t?  +  1.25)  (  s^w  -  mins-^ui )  (13) 

weT  ~  ^  '  \  weT        J  ^     ' 

and 

max s^w  —  s^w  >  I )  (  s^w  —  mills'^ u; )    .  (14) 

w€T  ~  Vs. 5(9  + 1.25/  V  weT        J 


3     Complexity  of  Computing  a  Feasible  Solu- 
tion of  Gp 

In  this  section  we  present  and  analyze  algorithm  FEAS  for  computing  a  feasible 
solution  of  Gp  using  the  barrier  method  in  equation-solving  mode.  The  output 
of  algorithm  FEAS  will  be  a  point  x  that  will  satisfy  .4x  =  6,  x  G  P,  as  well  as 
several  other  important  properties  that  will  be  described  in  this  section.  The 
computed  point  x  will  also  be  used  to  initiate  algorithm  OPT,  to  be  presented 
in  Section  4,  that  will  start  from  x  and  will  then  compute  an  e-optimal  solution 
of  Gp. 

Algorithm  FEAS  will  employ  the  barrier  method  in  equation-solving 
mode  to  solve  the  following  optimization  problem  denoted  by  Pi: 

Pi  :    t'  :=  maximum^  (  0    t 


[lb) 


s.t.  Az  =  {b-  Ax^)e 

z  +  dx'  -  tx°  e{e-  t)P 

0  <  1 

e  >  t 

t  >  -2 

\\4  <  1 

where  x""  and  x"  are  the  pre-specified  reference  point  and  interior-point,  re- 
spectively, and  where  we  use  the  notation  aP  as  follows: 


{x  G  X  I  X  =  aw  for  some  iv  E  P}    if  q  >  0 
aP  :=  {    recP  if  q  =  0  (16) 

/)  if  Q  <  0 
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and  where  recP  denotes  the  recession  cone  of  P. 

Note  that  Pi  is  an  instance  of  the  optimization  problem  OP  of  Section 
2,  with  w  —  {z,t,9),  c  =  (0, 1,0),  etc.  We  will  employ  the  barrier  method  in 
equation-solving  mode  to  solve  Pi  for  a  feasible  solution  (I,  t,  9)  with  objective 
value  c^w  =  J  :=  0,  i.e..  for  a  feasible  solution  (i,  i,  6)  of  Pi  for  which  i  —  0.  We 
will  then  convert  this  solution  to  a  feasible  solution  of  G p  via  the  elementary 
transformation: 


+  x^ 


(17) 


(where  the  algorithm  will  ensure  that  9  >  Q  and  so  (17)  will  be  legal).  Note 
that  if  (i,  0, 9)  is  feasible  for  Pi  and  0  >  0,  then  it  is  straightforward  to  verify 
that  X  from  (17)  satisfies  Ax  —  b,x  €  P. 

In  order  to  solve  Pi  for  a  solution  (zJ,9)  =  {z,0,9),  we  first  must 
construct  a  suitable  barrier  function  for  Pi.  Let  Si  denote  the  feasible  region 
of  Pi.  namely: 

Si  :=  {{:J,9)  I  .4.'  =  [b  -  Ax'')B,  z  +  9x'  -  tx^  G  {9-t)P,9  <  l,9>t,t>  -2.  ||;||  <  l} 


(18) 


and  consider  the  barrier  function: 


F{zJ.B)  :=  -ln(?+2)-ln(l-0)+f||  \\{z)+400 


Fp 


z  +  9x'  -  tx° 


2dp\n{e-t) 
(19) 


Define: 


d:=2  +  i?||  II  +  SOOdp  . 


Then  from  the  barrier  calculus,  and  in  particular  from  Proposition  5.1.4  of  [5], 
we  have: 


Proposition  3.1   F{z,t,9)  is  a  -d -self- concordant  barrier  for  S^.  | 


Note  that  1^  =  0  Ci)p  +  d^  A. 


We  will  initiate  the  barrier  method  at  the  point  {z,t,9)°  :=  (0,  -1.0). 
Thus  our  algorithm  for  finding  a  feasible  point  of  Gp  is  as  follows: 

Algorithm  FEAS:  Construct  problem  Pi  and  the  barrier  function  (19).  Us- 
ing the  starting  point   {z.t,9)^   :=   (0.-1,0).  apply  the  barrier  method,   in 
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equation-solving  mode,  to  compute  a  feasible  solution  (i,  t,  0)  of  P\  that  sat- 
isfies f  =  5  =  0.  If  such  a  solution  is  computed,  then  compute  x  using  (17). 

We  now  examine  the  complexity  of  algorithm  FEAS.  To  do  so,  we  first 
bound  the  symmetry  of  5i  at  the  point  {z,  t,  ^)^: 


Proposition  3.2   {z,t,6)^  :=  (0,  -1,0)  is  a  feasible  solution  of  Pi,  and 
(,        ,,0   ^x       mm{dist(xO,aP),l} 

The  proof  of  Proposition  3.2  is  deferred  to  the  end  of  the  section. 

We  will  also  need  the  following  relationship  between  the  optimal  value 
of  Pi  and  g  : 


Proposition  3.3  Let  t'  denote  the  optimal  value  of  Pi.   Then 
(min{dist(x°,6>P),l})  •  5  <  -^  <  g  [g  +  I  +  \\x^  -  x' 

The  proof  of  Proposition  3.3  is  also  deferred  to  the  end  of  the  section. 

We  next  examine  the  range  of  the  objective  function  value  of  Pi.  Be- 
cause 

2"  =  max{t  I  {z,t,e)  e  5i}  =  r  <  1 

(from  the  constraints  i  <  ^  <  1  of  Fi)  and 

z'  =  mm{t  I  {z,t,9)  e  Si}  >  -2  , 
we  have 


P:=max{^  |  {zj,0)  G  Si}  -  mm{t  \  {zj.,9)  E  Si}  <  3  .  (20) 

Finally,  observe  that  z'  <  -I  (from  Proposition  3.2),  and  so  with  (5  :=  0  we 
have: 

min{."  -  (5,  S-z'}>  mm{t\  1}  =  T  >      ,    ^  ,  ^  |,   p ^  >  (21) 

g{g  +  l  +  \\x°  -  x'^W) 

where  the  last  inequality  is  from  Proposition  3.3.  Combining  Propositions  3.2 
and  3.1  as  well  as  (20)  and  (21),  and  using  (10)  and  Proposition  A.l  of  the 
Appendix,  we  obtain  the  following: 
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Theorem  3.1  Under  Assumption  A.  algorithm  FEAS  will  compute  a  feasible 
solution  {z.t,9)  of  Pi  and  by  transformation  a  feasible  solution  x  ofGp,  in  at 
most: 

1 


O  Udp  +  J||  II  In  Lp  +  ^ii  II  + 


\x'-x^\\+g 


min{dist(xO,c)F),l} 
iterations  of  Newton's  method.  | 


Given  the  output  {zj.,9)  and  the  transformed  point  x  given  in  (17) 
from  algorithm  FEAS,  define  the  following  set: 

52  :=  Ix  e  X  I  Ax  ^b,x  eP,  \\x  -  x'W  <  i}   •  (22) 

The  following  characterizes  important  properties  of  (i,  t,  9)  and  x  that  will  be 
used  in  the  analvsis  in  Section  4: 


Lemma  3.1  Suppose  that  Assumption  A  is  satisfied,  and  let  (i,  t,  9)  and  x  be 

tfie  output  of  algorithm  FEAS.  Then 

(i)  X  e  S2,  and  sym(I-,  52)  >  g^^^^^a 

(ii)  i  <  (3.57? +  2.25)  5 

(iti)  ||x-x''||  <  (3.51^  +  2.25)5 

M    1  -  ^  >    3.5^+2.25 

fv)  1  -  ll^ll  >  ^- 

I  ^/    J-  11-11   ^    3.5t3+2.25 

(vi)  Let  [x,  f)  be  an  optimal  solution  of  (4).    Then 


PnB{x\i 


B  U        '^^^'^'\      \  c 
V     (3.5t?  +  3.25)    -5/ 

Proof  of  Lemma  3.1:  It  follows  from  Proposition  3.3  that  f  >  0,  and  so  from 
the  barrier  method  the  point  {z,t,9)  =  {z,0,9)  will  satisfy  Az  =  {b  —  Ax'')9, 
z  +  9x'  e  mt{9P),  9  <l,9  >0,  and  ||i||  <  1.  Then  ^  >  0  validates  (17),  and 
also  Ax  =  b,  X  E  P,  and  ||x  -  x^\\  =  i||z||  <  i  whereby  we  see  that  x  €  52- 
This  proves  the  first  assertion  of  (i). 
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Let  Ti  :=  Sx  n  {{z,t,e)\  t  -  0}  where  recall  that  Si  is  the  feasible 
region  of  Pi,  see  (18),  and  let  T2  :=  Si  n  {{z,t,9)\  t  =  0,9  =  6}.  Then  from 
(11).  (2.iJ)  =  {z,Oj)  will  satisfy 

sym{{z,iJ),Ti)> 


3.5?^ +  1.25 


Furthermore,  since  T2  is  the  intersection  of  Ti  with  an  afRne  space  passing 
through  {z,i,9),  then  it  also  follows  that 

sym{{zJJ),T2)> 


3.b{)  +  1.25 


Also,  the  affine  transformation  (2,  t,  0)  h->  (|  +  x'')  maps  T2  onto  52  (see  (22)) 

and  maps  {z,i,9)  to  x,  and  since  symmetry  is  preserved  under  afRne  transfor- 
mations, it  follows  that 

sym(x,52)  >  — — --j— -     ,  (23) 

6.0V  +  1.25 

completing  the  proof  of  (i). 

Let  S  :=  min{||x  -  x''||  |.4x  =  6,  x  €  P},  and  note  that 

1  1  1 

max    9  =  T-. — r  > 


(2,f,0)eri         max{d',  1}       max{5,l}       g 

since  g  >  6  and  5  >  1.  Noting  as  well  that  m.m(^^t,6)eTi  ^  =  0,  it  follows  from 
(13)  that 

(3.5t9  +  1.25)^    =     (3.5'i}  +  1.25)(^-min(,,,,o)gri^) 

>  max(,_t,0)eTi  9-9 

>  '--9    , 
—     9  ' 

and  rearranging  yields  i  <  (3.5i^  +  2.'2b)g.  This  proves  (ii).  (iii)  then  follows 

since  ||j  -  x'H  =  ^  <  i  <  (3.5i9  +  2.25)5.  Noting  that  maX(,,t,(,)6Ti  ^  <  1  and 
min(^  (gjgT-j  ^  =  0,  it  follows  from  (14)  that 

l-9>m&X(^^tfl)eTi9-9    >     ( 3,5^+1.25  )(^  "  ^^t^{z  ,t  ,0)€T,  0) 


3.5i9+1.25       ' 

and  rearranging  yields  1  -  ^  >       „ '       ,  which  proves  (iv). 
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We  now  prove  (v).  Given  z,  there  exists  z  E  A'*  satisfying  \\z\\,  =  1 
and  z^  z  =  ||i||,  see  Proposition  A. 3  of  the  Appendix.  Then 

1  -  pll     >    max(^,t,o)6T,  z^z  -  z^z 

^    ( 3.5^+1.25  )(^^^  ~  min(,,,.tf)gri  z'^z) 

where  the  second  inequality  above  is  from  (14),  and  rearranging  yields  1  - 

IMI  <  (3.5^+2.25)'  proving  (v). 

In  order  to  prove  (vi),  we  will  use  the  following  claim: 

there  exist  (f ,  f)  satisfying  Ax  =  b,  B{x,  r)  C  P,  \\x  —  x^\\  +  f  <  -     ,     (24) 

9 

and 

min{r,  1} 

'-5(3.01^  +  3.25)     '  ^^'' 

where  (i,r)  is  an  optimal  solution  of  (4).  We  assume  for  the  rest  of  the  proof 
without  loss  of  generality  that  r  <  1  and  so  minjr",  1}  =  f- 

Before  proving  (24)  and  (25),  we  use  them  to  prove  part  (vi)  of  the 
Lemma.  From  (24)  and  (25)  we  have  x  E  S2  and  so  from  (23)  x^  :=  x  — 
^3  5tf+2  25K-^  -  x)  €  52.  Rearranging  this,  we  obtain 

_^  3.57^  +  2.25    1  1  ,     . 

^      3.5?^ +  3. 25^    ^  3.5i5  +  3.25^  ^"  ' 

and  so  .f  is  a  convex  combination  of  x'  e  S2  and  x  G  52.  It  then  follows  from 
(26)  that  B  (x,  3-5^^:325)  C  \P  n  B  fx'",  i)],  which  combined  with  (25)  proves 
(vi). 

It  remains  to  prove  (24)  and  (25).  We  consider  two  cases: 

Case  1:  {g  +  7")^  <  1.  Let  x  =  x  and  f  =  r.  Then  Ax  =  5,  5(x,  f)  C  P,  and 


X' 


x||  +  r  =  ||x  -  x'"||  +  r  <  g  +  r  <  i.  Furthermore,  f  =  r  > 


and  so  (24)  and  (25)  are  proved. 

Case  2:  {g  +  f)9>l.  Let  x  =  (1  -  /?)x  +  /3x,  where 


-.     i  ciiuiiv^xiiiwiv.,    ,  _    g(3.5i3+3.25) 


P  = —^ ,  (27) 

l  +  (3.bi)  +  2.2o){e{g  +  r)-l) 
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and  let  f  =  (if.  Then  j3  G  [0,  1],  and  so  Ax  =  5,  and  B{x,f)  C  P.  Also, 
||f_x'"|| +f    =     \\(l  -  P){x  -  x"-)  +  p{x  -  x')\\  +  pf 

<  {I- p)\\x-x'\\+P\\x-x'\\+ I3f 

<  [l  -  13)^  +  Pg  +  Pf 

<  (l-^)(J)(|f^)+^5  +  /3r 
_     1 

where  the  last  inequality  follows  from  part  (v)  of  the  lemma,  and  the  last 
equality  follows  directly  from  (27).  Also, 

f  ^  I3f  = -. 3 > 


1  +  (3.5(^  +  2.25){9{g  +  f)  -  1)  "  5(3.5i3  +  3.25) 
since  ^  <  1,  r  <  1,  and  g  >  I-  This  then  proves  (24)  and  (25)  in  this  case.| 

Proof  of  Proposition  3.2:  Note  first  that  {z,t,ef  =  (0,-1,0)  is  feasible 
for  Pj.  It  suffices  to  show  that  if  (0  +  d,  -1  +  a,  0  +  (5)  is  feasible  for  Pi, 
then  {0  -  l3d,  -1  -  a/3,  0  -  (36)  is  feasible  for  Pi,  where  /?  =  3+2||J"p-x'-||'  ^^^ 
where  r  is  given  by  (3).  Note  that  ^  <  ^  <  5.  Since  (0  +  d,  -1  +  a,  0  +  (5)  is 
by  presumption  feasible  for  Pi,  then  Ad  =  {b  -  Ax^)6,  d  +  6x^  +  (1  -  Q')x°  G 
((5  +  1  -  a)P,  6  <l,S>-l  +  Q,  \\d\\  <  1,  -1  +  a  >  -2,  and  it  follows  that 

-2  <  5  <  1     and      -  1  <  a  <  2  .  (28) 

Let  {z,t,e)  :=  (-f3d,-l  -aP,~P6).  Then  Az  =  {b  -  Ax')9,  and  ||z||  <  1 
since  /3  <  1.  Also  9  =  -pS  <  2,5  <  1  from  (28)  and  /?  <  |.  Next,  notice  that 
e  -  t  =  -P5  +  1  +  ap  ^  1  -  P{S  -  a)  >  I  -  ^{1  +  1)  >  0  from  (28)  and  /?  <  i. 
Also,  t  =  —I  —  aP  >  -2  since  a  <  2  and  /3  <  |.  It  remains  to  prove  that 
z  +  Ox''  -  tx°  e  {9  -  t)P.  To  see  this,  note  first  that 

\\- pd-8p{x' -x'')\\-aP  +  5p    <    p{\\d\\  +  2\\x'  -x^)  +  2p    (from(28)) 

<    pI?,  +  2\\x' -x°\\) 

=     r  . 


Then 

0       II 

-d- 

-  J(x'' 

-x')\\     < 

^(l  +  2r-x«||) 

(from(28)) 

H-a;3-(5^  II 

= 

,,      l+2||x'--x«||      ^ 
^3-2r+2||x'--iO||; 

< 

r 

(since  r  <  1). 

Therefore  x^  +  ^_^^^_^a(-c/  -  S{x''  -  x°))  G  P  from  (3),  and  rearranging  yields 
-pd-  p5x'  +  {l  +  ap)x°    e    {l  +  ap-6P)P, 


GEOMETRY-BASED  COMPLEXITY  IG 

which  is  the  same  as  2  +  9x^  -  tx^  £  {6  —  t)P.  This  shows  that  {z,t,0)  is 
feasible  for  Pi,  and  so  sym((c,  f,0)°,    Si)  >  /3  as  desired.| 

Proof  of  Proposition  3.3:  Let  (i,  f)  be  an  optimal  solution  of  (4),  and  note 
fruiu  (4)  that  we  can  assume  that  r  <  1.  From  Assumption  A,  r  >  0.  Define 
the  following: 

3^-^'  ^  ^  3f  g_  1  ,29) 

max{||i  -  x'"||,  1}'  max{||f  -  x''||,  1} "  max{||x  -  .r''||,  1} 

where 

^  =   3+l  +  ||:,r_^0||-  (30) 

Then  /?  <  1,  since  in  particular  g  >\,  and  from  (29)  we  have  Az  —  {b-  Ax'')6, 
e  <l,  -2  <t  <9,  and  \\z\\  <  1.  If  (zJj)  also  satisfies 

z  +  9x'  -tx°  e{0-t)P,  (31) 

then  {z,t,9)  is  feasible  for  Pi,  whereby 

f>t=  ^' >  ^  = ^ (39) 

max{||x-x'-||,l}  -  5       g{g  +  I  ^  \\x' -  x^\\)  ' 

proving  the  second  inequality  of  the  proposition.  Therefore,  to  prove  the 
second  inequality  of  the  proposition,  we  must  show  (31).  Xote  first  that 

^  =  5  +  1  +  ||x^  -  x°||  >  ||.r  -x'\\+r  +  \\x'  -  x°||  >  r  +  \\x  -  x°||,       (33) 

and 

^l|x  -A\-  Y^Wi  -  A\  =  y-Wi  -A\<  r,  (34) 

where  the  last  inequality  above  follows  from  (33),  and  9  >  t  (since  /?  <  1  and 
f  <  1),  and  so 

z  +  9x'-tx°        ^  t     r-         0,       r^ 

9-t      =^  +  rrt(--AeP, 

since  from  (34)  we  have  ^||f-x°||  <  r  and  B{x,f)  C  P.  Therefore  z  +  9x^  — 
tx"  G  (^  -  t)P,  and  we  have  proven  the  second  inequality  of  the  proposition. 

To  prove  the  first  inequality  of  the  proposition,  let  {z*,t',9'')  be  any 
optimal  solution  of  Pi,  and  note  from  (32)  that  t*  >  0.  Therefore  9*  >  0,  and 
define 

x  =  -  +  x  ,    r  =  —  ,  (3d) 

where  r  is  defined  in  (3).  Then  .4x  =  b,  and  for  any  d  satisfying  ||d||  <  1  we 
have  x°  +  Td  e  P  from  (3).  Also,  z'  +  9*x''  -  rx°  €  (r  -  r)P.  If  ^  >  t% 
then 

^         [9*  -t'\    Z*  +9'X'  -t'x°         fr\,o  n         r, 
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If  6**  =  r,  then  z'  +  O^x'  -  t'x°  £  recP,  and  so 

X  +  rd  = h  (x"  +  Tci)  e  P. 

In  either  case,  x  +  rci  G  P,  and  so  5(x,  r)  C  P.  Therefore 

maxjll.c'"  -  xll,  1}  f  lla:'"  —  xll    1 

9  <  ^"    J     ,''     -^  =  max    ^ ^,  - 

mui{r,  1)  y        r  r 

since  ?-  <  1.  Now 

I  _  e*       1 

r       t*r  "~  t*T 

and  11^—^   =    1^    <    p7,  so  5   <    p;:,  proving  the  left  inequality  of  the 
proposition-! 

Certain  ideas  and  constructs  used  in  the  results  of  this  section  arose 
from  or  were  inspired  directly  from  Section  3  of  Renegar  [7],  including  the  idea 
of  solving  phase-I  by  using  the  barrier  method  in  equation-solving  mode,  trans- 
forming to  the  original  problem  via  an  elementary  projective  transformation, 
and  establishing  key  properties  of  the  output  of  the  algorithm  (upper  bounds 
on  norms  and  lower  bounds  on  distances  from  constraints)  using  symmetry 
properties  of  the  output  of  the  barrier  method. 


4     Complexity  of  Computing  an  e-optimal  So- 
lution of  Gp 

In  this  section  we  present  algorithm  OPT  for  computing  an  e-optimal  solution 
of  Gp  initiated  at  the  point  i,  using  the  barrier  method  in  optimization-mode, 
where  x  is  the  output  of  algorithm  FEAS. 

Using  X  as  a  starting  point,  we  will  modify  Gp  slightly  by  adding  a  level- 
set  constraint  of  the  form  "c^x  <  c^x  +  s"  to  Gp  for  some  suitably  chosen 
positive  scalar  offset  s  which  will  then  render  the  point  x  in  the  interior  of  the 
half-space  generated  by  the  constraint  c^x  <  c^x-\-s.  The  question  then  arises 
as  to  how  to  choose  the  offset  s.  One  would  think  that  s  should  be  chosen 
proportional  to  the  norm  of  c  : 

||c|L  ;=  m.s.x\c^w  I  ||u'||  <  l[   .  (36) 

However,  because  the  objective  function  c^x  of  Gp  differs  only  by  a  constant 
from  the  modified  objective  function  (c  —  A^tx)'-^x  over  the  feasible  region  of 
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Gp  for  any  given  value  of  tt,  we  must  be  mindful  of  the  equations  Ax  =  b. 
From  this  perspective,  it  is  natural  to  choose  s  proportional  to: 


s  :=  max  |c^ a'  I  ||u'||  <  l,-4u'  =  o|  ,  (371 


and  note  s  is  the  maximum  objective  value  over  the  unit  ball,  "reduced"  by  the 
subspace  constraints  Aw  —  0,  and  so  s  is  the  norm  of  the  linear  functional  c^x 
over  the  vector  subspace  of  solutions  to  Aw  =  0.  However,  even  for  otherwise 
computationally  tractable  norms  such  as  the  L^  norm  in  JR",  the  computation 
of  s  is  not  trivial;  in  fact  its  computation  is  a  linear  program  for  the  Zoo  norm. 
We  therefore  will  instead  use  the  information  inherent  in  the  barrier  function 
F\\  ii(-)  for  the  unit  ball  as  a  proxy  for  the  ||  •  ||  in  constructing  the  offset  s.  Let 
H{Q)  denote  the  Hessian  matrix  of  F\\  ||(-)  at  x  =  0,  and  define: 

52  :=  max  [c^w  \  Aw  =  0,  w'^H{{))w  <  l}  ,  (38) 

and  note  that  $2  admits  a  closed  form  solution  when  rank(.4)  =  m  : 


S2  =  s]c'^H{{))-'c  -  c^//(0)-^4^  iAH{0)-'A'^)-'  AH{0)- 


It  will  be  convenient  for  our  purposes  to  determine  s  proportional  to  S2  as 
follows: 

and  we  consider  the  following  amended  version  of  Gp: 


Ps  :    z*  :=  minimum-c    c^x 

s.t.  Ax  =  b 

xeP 


(40) 


(Fx  <  c^x  +  s 


Note  that  since  x  is  feasible  for  Gp,  then  x  is  also  feasible  for  Pj, 
and  Pg  and  Gp  have  the  same  optimal  objective  function  value  and  the  same 
set  of  optimal  solutions.  (The  idea  of  solving  phase-H  by  adding  a  level  set 
constraint  of  the  objective  function  was  used  by  Renegar  [7],  but  without  an 
explicit  construction  for  computing  the  offset  s.) 

In  order  to  apply  the  barrier  method  (in  optimization  mode)  to  compute 
an  e-optimal  solution  of  Fj,  we  need  to  specify  the  barrier  function  to  be  used. 
The  obvious  choice  is: 
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F{x)  :=  Fp{x)  -  In  (Jx  +  s-  c^x)   ,  (41) 

whose  complexity  value  is  at  most  i3p  +  I. 

It  is  easily  seen  that  s  >  0,  and  that  s  >  0  except  when  the  objective 
function  c^x  is  constant  over  the  entire  feasible  region  of  Gp,  in  which  case  x 
is  then  an  optimal  solution  of  Gp.  In  light  of  this  observation,  the  algorithm 
for  computing  an  e-optimal  solution  of  Gp  is  as  follows: 

Algorithm  OPT:  Compute  s  and  construct  problem  Pj  and  the  barrier  func- 
tion (41),  using  (38)  and  (39).  If  s  >  0,  then  using  the  starting  point  x  (where 
X  is  the  output  of  algorithm  FEAS),  apply  the  barrier  method,  in  optimization 
mode,  to  compute  an  e-optimal  solution  x  of  Pg.  Otherwise,  s  =  0.  and  x  is 
an  optimal  solution  of  Gp  and  no  further  computation  is  required. 

The  rest  of  this  section  is  devoted  to  proving  the  following  complexity 
bound  for  algorithm  OPT: 

Theorem  4.1    Under  Assumption  A,  and  starting  from  the  point  x  computed 

by  algorithm  FEAS,  algorithm  OPT  will  compute  an  e-optimal  solution  oj  Gp 
in  at  most: 

oi^fh'\n  {g^-  D,  +  dp  +  d\\  y -h  max  | -,  1 1 J  J 

iterations  of  Newton's  method.  \ 

Remark  4.1  Note  that  we  could  replace  s  by  \\c\\,  in  the  iteration  bound  of 
Theorem  4-1-  since  s  <  ||c||,. 

We  begin  the  analysis  underlying  the  proof  of  Theorem  4.1  by  relating 
the  two  quantities  s  and  s: 


Proposition  4.1    I  ^ 


s  <  s  <  s 


Proof:  Since  Fy  ||(x)  is  a  -i^n  y-self-concordant  barrier  for  B{0, 1)  =  {x  |  ||x||  < 
1},  then  F{x)  :=  f||  ||(x)  -  F\\  \\{-x)  is  a  2d\\  ||-self-concordant  barrier  for 
B(0, 1),  whose  analytic  center  is  x'^  =  0,  and  note  that  the  Hessian  of  F{x) 
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at  X  =  0  is  2//(0)  where  H{x)  is  the  Hessian  of  Fy  ||(-)  at  x.    Then  from 
Proposition  2.3.2  of  [5]  it  follows  that 


{x  I  ^x^{2H{0))x  <  l}  C  5(0, 1)  C  [x  \  yx^(2/f(0))x  <  3(2,9||  y)  +  l}  . 

m 

From  this  it  then  follows  that  4=52  <  s  <  — ^| — S2,  and  therefore  the  result 
follows  from  (39). | 

We  will  make  use  of  the  following  lemma  which  bounds  the  growth  of 
the  level  sets  of  Gp. 


Lemma  4.1  Suppose  that  x  is  a  feasible  solution  of  Gp  satisfying  (Fx  <  a 
for  some  given  level  a,  and  further  suppose  that  B{x,f)  C  P  for  some  r  >  0. 
Suppose  that  Q  satisfies: 

Q>  max  [\\x -x\\\  Ax  =  b,x  e  P,c'^x  <  a]   .  (43) 

Then  for  all  t  >  0  and  for  all  x  satisfying  Ax  =  b,x  £  P,c^x  <  a  +  t,  the 
following  inequality  fiolds: 

2t 


\x  -  x\\<Q\l  + 

V        s  ■  r 


Proof:  Given  the  hypotheses  of  the  lemma,  suppose  that  x  satisfies  Ax  =  6, 
X  e  P,  and  c^x  <  a  +  t.  Define  s'  :=  c'^x  —  a.  If  s^  <  0,  then  c^x  <  a,  and 
therefore  ||x  —  x\\  <  Q  <  Q{1  +  |4),  proving  the  result.  Suppose  instead  that 
s^  >  0,  and  define 


s^  \  ,.       ,,,       /     a  —  c^x  +  rs 


w  :=     ^ TT r     i^  -fc)+\  — r^ X,  (44) 

\a  —  c^  X  +  rs  +  s^  J  \a  —  c^  x  +  rs  +  s'-  J 

where  c  G  argmax„{c^u  |  Av  =  0,  ||u||  <  1}.  Then  ||c||  <  1,  and  c^c  =  s. 
Also  X  -  fc  e  P,  A(x  -  fc)  =  b,  Ax  =  b,  X  e  P,  and  so  it  follows  from  (44) 
that  Aw  —  b,  w  E.  P,  and  c^ w  =  a,  since  (Fc  =  s.  Therefore 

Q    >     ||u.  -  f  11  =  11  f 7"-^i'-^  i)c+(    "-/"f  t/^^  1 )  (j  -  f  )|| 

^       —       II                 II          II   \a-c'  x+TS  +  s^  J             \a-c'  x  +  rs  +  s'  J   ^  ' '' 

\       II  T  _   ^11/     a-c^i+fs     \  _  fs^ 

^"^  X  .Al  7"-       --         i;  7'---,! 

—       II  II  \  a  — c' x  +  r5  +  s'  '  a  —  c'  x-trs+s^ 
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Therefore 


\\x-x\\    <    Q{i  +  ^z^)  +  ^z^, 

(45) 

However,  as  noted  above,  x  -  re  G  P  and  A{x  -  re)  =  6,  and  c^{x  -  re)  < 
e^x  <  a,  and  so 

Q  ^  \\x  —  {x  —  fc)\\  =  f. 

Therefore  from  (45)  we  have 

||x-x||     <    Q(l  +  fl)  +  4(f) 

<     Q(l  +  ll)^ 
since  x^  —  c^x  —  a  <  f.| 


Lemma  4.2  Suppose  that  D^  is  finite.   Then  there  exists  (x,  f)  satisfying 
Ax  =  b  ,    B(x,  r)  C  P  ,    c^f  <  2:*+e,    and  f  >  min  {r,  1}  min  <  1, 


c^x  —  z 

(46) 

where  (x,r)  is  an  optimal  solution  of  (4)- 

Proof:  Without  loss  of  generality  we  can  assume  that  r  <  1.  Suppose  first 
that  c^x  <  z'  +e.  Then  setting  x  =  x  and  f  =  r,  we  have  c^x  <  2'  +  e,  Ax  —  b, 
B{x,  r)  C  P,  and  f  —  f  =  f  min{l,  ^,,^^_..  },  proving  the  result.  Suppose  instead 
that  c^x  >  z*  -\-  e,  and  let 

X  =  Ax  +  (1  -  A)x'     ,    f  =  Ar     , 

where 


c'^x 


and  x'  is  an  optimal  solution  of  Gp  (x*  is  guaranteed  to  exist  since  D^  is 
finite  by  hypothesis).  Then  A  G  [0, 1],  and  so  x  satisfies  ,4x  =  b,  x  &  P,  and 
by  construction  of  A  we  have  c^x  =  z*  +  e.  Furthermore  B{x,  r)  C  P,  and 
r  =  Ar  =  ^T~L.-  =  ^ ™iri  1 1 ,  ^Ti_,.  ]  1  proving  the  result .| 

Define  53  to  be  the  feasible  region  of  P^,  namely 

53  :=  |x  I  .4x  =  6, X  6  P, c^x  <  c^x  +  s\   . 
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Lemma  4.3  Suppose  that  x  E  S3.   Then 

\\x  -  x||  <  ^ 
where 
h  :=  3A  +  (3.5(?  +  2.25)  g+4gD,  {g  +  D,)  [(3.5^  +  2.25)  5  +  A  +  679||  y  +  l]  max  |  -,  1 


Remeirk  4.2  h  is  bounded  from  above  by  a  polynomial  m  5,i?p,t'||  \\,De,,  arid 
max{f,l}. 

Proof  of  Lemma  4.3:  Suppose  that  x  €  S3,  and  let  (x,  f)  be  as  described  in 
Lemma  4.2.  Then 

||x  -  x||     <     ||x  -  ill  +  llx  -  x'"||  +  llx""  -  x|| 

(47) 
<     ||x-x||  +  A  +  (3.5i?  +  2.25)5 

where  the  last  inequality  uses  (6)  and  Lemma  3.1.  It  thus  remains  to  bound 
Ijx  -  x||.  To  this  end,  we  will  invoke  Lemma  4.1  with  a  =  z*  +  e.  Then 
Q  :=  2De  satisfies 

maxi{||x  -  x||  I  .4x  =  6,  x  6  P,  c^x  <  a} 

<  maxj{||x  -  x'll  +  IJx'"  -  x||  |  Ar  =  b.  x  e  P.  c^x  <  z'  +  e} 

<  ■2D,  =  Q  , 

and  so  x,  f,  a,  and  Q  satisfy  the  hypotheses  of  Lemma  4.1.  Now  let  t  := 
[c^x  -i-  s  —  a]'^,  and  so  a  +  i  >  c^x  +  s.  Then  if  x  6  53,  x  also  satisfies  Ax  =  b, 
X  €  P,  c^x  <  a  +  t,  and  so  from  Lemma  4.1  we  have: 

||x-x||     <     Q(l  +  |) 


2  A  +  ifi 

(48) 

5  min{f,l} 


<    2D,  +  ■  ^^/^,  maxfl.  ^^^^^^^j  (from  Lemma  4.2 


<    2D,  +  '^'J'^''^,-':^^"'  maxll,  '^^^^} 

—  '  smin{r,l)  I     '  (         J 
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But  now  observe  that 

c^x  —  z*  +  s    =    (Fx  —  c^x*  +  s  (where  x*solves  Gp) 

<  s(||x  -  j;'"||  +  ||i-*  -  x""!!) +  s  (from  Proposition  A. 2) 

<  s(||.f  -  x'll  +  ||x'  -  x""!!)  +  s(6i?||  II  +  1)      (from  Proposition  4.1) 

<  s  [{3M  +  2.25)5  +  A  +  6i?||  y  +  l] 

.  (49) 
where  the  last  inequality  uses  (6)  and  Lemma  3.1.  We  also  bound  c^x  —  z* 
using  Proposition  A. 2: 

c^x  -  2*  =  c^x  -  Jx*  <  s{\\x  -  x'W  +  ||x*  -  x'll)  <  ~s[g  +  D,)  .         (50) 

Substituting  (49)  and  (50)  into  (48)  we  obtain 

II      _    -II       <r      on    u.  -*g4(3.5i?+2.25)9+D,+6iJ||   ||  +  1]  max{l/"'°+^'' } 
IK"        •'^ll       -      ^-^t  +  min{f,l} 

<    2D,  +  4D,g[{3.5d  +  2.25)g  +  D,  +  6d^\  y  +  l]{g  +  D,)  max  {l,  f  }   . 

(51) 

Finally,  substituting  (51)  into  (47)  we  obtain  the  desired  bound.| 


Lemma  4.4 


<  (3.5t9  +  2.25)^ /t 


sym  (x,  53) 
where  h  is  the  quantity  defined  in  Lemma  4-3. 

Proof:  Let  /?  :—  (3  5^  >  2  251-/1 '  ■^^'-  ^"^  ^  satisfy  x  +  v  G  S3,  we  must  show  that 
X  -  Pv  e  53.  To  do  so,  we  must  show  that  .4(x  —  I3v)  =  b,  x  —  /3v  &  P,  and 
c^{x  -  j3v)  <  c^x  +  s.  Note  that  x  G  53  and  x  +  v  G  53  imply  that  Av  =  0, 
and  so  A{x  -  pv)  =  b.  Also,  from  Lemma  4.3,  we  have  ||i;||  <  /;,.  And  with 
Q  :=  min  |l,  ^~}r  ]  where  z,6  are  part  of  the  output  of  algorithm  FEAS,  we 
have  x  +  Qv  E  P  (since  x  +  v  E  P  and  a  G  (0, 1]).  Observe  that 

rn  M  .„  T  Pll  l-||i|l  1 

X  +  av  -  x'\   <    X  -  xn  +ah<  K^-\- 


e         e        e 
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and  so  x+av  €  52  {see(22)).  Then  from  Lemma 3.1  we  have  •^~3  5,^°i25^  ^  -^2, 
and  note  that 


Q  =  mm 


(l.^P)     >    minll.--— ^— tjI     (from  Lemma  3.1) 

{  Oh      )       —  \       (3.5<»+2.25)e/i  J       ^  ' 

^    "^^"{  1^^3.5^-2.25)}  (since  ^<1)        (52) 


A(3.5i9+2.25) 


Therefore  3^^^^  >  ^(3.5^12.25)^  =  '^'  ^nd  so  x-^t;  G  52,  whereby  i  - /3^;  G  P. 
Finally,  note  that 

(F{x-Pv)    <    c^x  +  Ps\\v\\ 

<  c^x + s3h 

<  (Fx  +  s^h        (from  Proposition  4.1) 

<  c^x  +  s    .  (since  fih  <  1) 

Therefore  x  —  3v  E  S3,  and  the  result  is  proved.| 

Proof  of  Theorem  4.1:  To  prove  the  theorem,  we  invoke  the  complexity 
bound  for  the  barrier  method  in  optimization  mode  stated  in  (8).  The  barrier 
F{x)  for  Ps  defined  in  (41)  has  complexity  value  d  <  dp  +  I  =  Oidp).  The 
starting  point  x  has  symmetry  bounded  by  Lemma  4.4: 

<{?,.od  +  2.2ofh    , 


sym(x,  53) 


this  bound  being  a  polynomial  in  5,  dp,  d\\  ||,  D^.  and  max{^,  1}.  see  Remark 
4.2.  The  range  R  of  the  objective  function  of  Pj  is  bounded  as  follows: 

R<c^x  +  s-  z'  <  s{{3.b0  +  2.25)5  +  D,  +  6!?||  y  +  1) 

from  (49).  Therefore  -  is  bounded  above  by  a  polynomial  in  max{j,  1},  dp, 
i?ll  ||,  g,  and  D^.  Combining  all  of  these  terms  and  using  Proposition  A.l  of 
the  Appendix,  we  obtain  the  complexity  bound  of 

O  [\/^ln  (g  +  D,  +  dp  +  d^  y  +  max  |  -•  1 1 
iterations  of  Newton's  method.| 
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5      On  Natural  Norms,  and  Condition-Number 
Complexity 

5.1     Two  Natural  Norms  on  X 

In  this  subsection  we  briefly  discuss  two  norms  on  X  that  arise  naturally  based 
on  x*^  and  P. 

The  First  Norm.  For  the  given  point  x"  G  intP,  define  the  set  Bj-o: 

B,o  :=  [P  -  x°]  n  [j°  -  P\  =  {v  I  x°  +  u  G  P,  x°  -  1'  G  P}  . 

Then  Bj.o  is  the  smallest  symmetric  set  B  for  which  x"  +  5  C  P.  Under  the 
assumption  that  P  contains  no  line,  B  will  be  compact,  convex,  and  contain 
the  origin  in  its  interior,  and  so  can  be  used  as  the  unit  ball  of  a  norm.  Indeed 
this  norm  is  constructed  as  follows: 


\v\\xO  :=    mmQ    a 

s.t.      x'^  +  ^v  e  P 


a 


X 


0 


^ve  P 


(The  norm  ||  •  ||jO,  in  either  expUcit  or  implicit  form,  appears  throughout 
much  of  the  analysis  in  [5].)  Under  ||  •  H^-o,  it  is  easily  shown  that  r(x°)  = 
dist(x°,9P)  =  1,  and  so  the  explicit  dependence  of  the  complexity  bounds  in 
Theorems  3.1  and  4.1  on  dist(x°,5P)  disappears.  Also,  we  can  construct  a 
barrier  function  for  the  unit  ball  B^o  using  the  barrier  Fp(-)  for  P: 

F||||(t.):=Fp(x°  +  i')  +  ^p(^°-^)> 
whose  complexity  parameter  'd\\  y  is  bounded  above  as  follows: 

■d\\  II  <  2dp  . 

Therefore  the  explicit  dependence  of  the  complexity  bounds  in  Theorems  3.1 
and  4.1  on  'd\\  y  disappears  as  well.  With  this  choice  of  norm,  then,  the  com- 
bined complexity  bound  of  the  algorithms  FEAS  and  OPT  becomes: 


O  (^p\n  (g  +  D,  +  i)p  +  max  |-,  l|  +  \\x°  -  x'\\\] 


iterations  of  Newton's  method. 
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(The  norm  ||  •  ||j.o  is  referred  to  as  a  generalization  of  the  Loo-norm 
because  in  the  case  when  P  =  3?"  and  x°  =  e,  we  recover  the  Loo-norm  as 
||u||^o  for  V  e  K".) 

The  Second  Norm.  The  second  norm  we  consider  is  constructed  using  the 
barrier  function  Fp(-)  for  P.  For  the  given  point  x^  G  intP,  define  the  norm 

||u||f,xO  :=  ^v'^Hp{x°)v  , 

where  Hp{x^)  is  the  Hessian  of  Fp(-)  evaluated  at  x  =  x^.  It  then  follows  from 
Theorem  2.1.1  of  [5]  that  5(.r°,  1)  C  P  and  so  dist(x^aP)  >  1.  Also, 

P||ll(^):=-ln(l-.^Lfp(x>) 

is  a  d\\  II  =  1-self-concordant  barrier  function  for  the  unit  ball  of  this  norm. 
Therefore  the  explicit  dependence  of  the  complexity  bounds  in  Theorems  3.1 
and  4.1  on  dist(x°,9P)  and  d\\  y  disappear,  and  like  the  previous  norm,  the 
combined  complexity  bound  of  the  algorithms  FEAS  and  OPT  becomes: 

O  (\fo~p\n  (g  +  D,  +  dp  +  maxl- ,l\  +  ||x°  -^ll)) 
iterations  of  Newton's  method. 


5.2      Relation  to  Condition-Number  based  Complexity 

Bounds 

In  this  subsection  we  indicate  how  the  condition- number  based  complexity 
bound  for  conic  convex  optimization  presented  in  (1)  can  be  obtained  as  a 
special  case  of  Theorems  3.1  and  4.1.  To  do  so,  assume  that  P  is  a  closed 
convex  cone  C ,  and  for  convenience  we  will  assume  that  C  is  pointed  and  has 
an  interior.  We  assume  that  we  have  a  ^c-self-concordant  barrier  Fc{-)  for  C, 
and  we  assume  as  in  [7]  that  the  norm  ||  •  ||  on  X  is  an  inner-product  norm 
||'t;||  :=  \/v'^v,  and  so  the  barrier  function 

F||  II (u)  :=  -ln(l  -v^v) 

is  a  d\\  II  =  l-self-concordant  barrier  for  the  unit  ball. 

Let  us  set  x''  :=  0  and  let  x°  G  intC  be  given,  and  assume  that  we  have 
rescaled  j"  so  that  ||x°||  =  1.  Then  from  Theorem  17  of  [3],  it  follows  that  g 
will  satisfv: 
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g<3C{d)- 


|.T°1 


dist(xO,5C) 
and  from  Theorem  1.1  and  Lemma  3.2  of  [6]  it  follows  that 

D,<C{df+C{d)-^ 
\\c\U 

where  C{d)  is  defined  liere  using  (2).  Then  under  the  hypothesis  that  e  <  ||c||,, 
the  combined  complexity  of  algorithms  FEAS  and  OPT  from  Theorems  3.1 
and  4.1  is: 


iterations  of  Newton's  method,  which  compares  favorably  to  (1). 
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APPENDIX 


Proposition  A.l:  If  a,b  >  0  theri  |  ln2  +  ^  (Ina  +  In 6)  <  lii(a  +  b).  If  in 
addition  a,b  >  1,  then  ln(a  +  6)  <  In 2  +  (Ina  +  In 6)  . 

Proof:  We  have  \/'2ab  <  \/ a^  +  6^  +  lab  —  a  +  b.  If  also  a,b  >  1,  then 
a  +  b  <  2  max{a,  6}  <  2  max{a,  b}  min{a,  b}  =  2ab.  The  results  then  follow  by 
taking  logarithms.! 

Proposition  A. 2:  If  x^,x'^  satisfy  Ax^  =  Ax'^  =  b,  then 


T„i       J' -^'i 


\c  x 


c^x^l  <  s\\x^  -  :r^||  <  s  ( llx^  -  xHI  +  llx^  -  x 


Proof:  From  the  definition  of  s  in  (37),  we  have 
Ic^x-i  -  c^x^l  =  \J[x'  -  x^)\  <  s\\x'  -  x^W  <  s  (\\x'  -  x'W  +  \\x^  -  x'W)   .| 


The  following  proposition  is  a  special  case  of  the  Hahn-Banach  Theo- 
rem; for  a  short  proof  of  this  proposition  based  on  the  subdifferential  operator, 
see  Proposition  2  of  [3]. 

Proposition  A. 3:    For  every  z  G  A',  there  exists  z  G  A'*  with  the  property 
that  \\z\\t  =  1  and  \\z\\  —  z^ z.  | 
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